Methods and systems for providing alerts to a driver of a vehicle via condition detection and wireless communications

ABSTRACT

Methods and system for alerting a vehicle driver via wireless communications. One or more conditions with respect to a vehicle are monitored. A change in the condition can be detected and a signal automatically and wirelessly transmitted to a computing device, wherein the signal is indicative of the change in condition(s). The driver is then alerted to the change in condition(s) in response to transmitting the signal to the computing device (e.g., a tablet computing device, smartphone, smartwatch, other wearable device, in-vehicle system, etc.) associated with the driver and/or a vehicle passenger.

CROSS-REFERENCE TO PATENT APPLICATION

This application is a continuation of U.S. patent application Ser. No.14/661,065 entitled “Methods and Systems for Providing Alerts to aDriver of a Vehicle Via Condition Detection and WirelessCommunications,” which was filed on Mar. 18, 2015, the disclosure ofwhich is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments are generally related to the field of electronic mobilewireless devices (e.g., smartphones, smartwatches, tablet computingdevices, laptop computers, etc.). Embodiments also relate to connectedcar technology and the deployment of electronic mobile wireless devicesin the context of automotive vehicles. Embodiments are additionallyrelated to the detection of external and internal conditions withrespect to a vehicle and the notification of alerts to a user regardingsuch conditions.

BACKGROUND OF THE INVENTION

The availability of on-board electronics and in-vehicle informationsystems has accelerated the development of more intelligent vehicles.One possibility is to automatically monitor a driver's drivingperformance to prevent potential risks. Although protocols to measure adriver's workload have been developed by both government agencies andthe automobile industry, they have been criticized as being too costlyand difficult to obtain. In addition, existing uniform heuristics fordriving risk preventions do not account for changes in individualdriving environments. Hence, technologies for understanding a driver'sfrustrations to prevent potential driving risks has been listed by manyinternational automobile companies as one of the key research areas forrealizing intelligent transportation systems.

Additionally, there is a need to monitor not only a driver's activity(e.g. driver inattention) but also to monitor the driver's activity withrespect to certain conditions, which may be external to the vehicle ormay occur within the vehicle. For example, a driver may be approaching ared light or the driver may be stopped at a red light and is looking athis cell phone or other distractions instead of being attentive to thetraffic light.

BRIEF SUMMARY

The following summary is provided to facilitate an understanding of someof the innovative features unique to the disclosed embodiments and isnot intended to be a full description. A full appreciation of thevarious aspects of the embodiments disclosed herein can be gained bytaking the entire specification, claims, drawings, and abstract as awhole.

It is, therefore, one aspect of the disclosed embodiments to provide foran improved method and system for alerting a driver of a vehicle of achange in condition external to or within the vehicle.

It is another aspect of the disclosed embodiments to provide a methodand system for issuing alerts to a vehicle driver and/or passenger viawireless communications.

The aforementioned aspects and other objectives and advantages can nowbe achieved as described herein. Methods and system are disclosed foralerting a vehicle driver via wireless communications. Such an approachcan include steps or operations for monitoring one or more conditionswith respect to a vehicle, detecting a change in the condition (orconditions), automatically transmitting a signal wirelessly to acomputing device, wherein the signal is indicative of the change incondition(s), and alerting the driver of the change in condition(s) inresponse to transmitting the signal to the computing device (e.g., atablet computing device, smartphone, smartwatch, etc.).

In some embodiments, the step or operation of alerting the driver to thechange in condition(s) can further include a step or operation ofproviding an audible alert via a speaker associated with the computingdevice, wherein the audible alert indicative of the change condition(s).In another embodiment, the step or operation of alerting the driver ofthe change in condition(s) can further include steps or operations forestablishing a wireless connection between the computing device and aradio system of the vehicle. And providing an audible alert from or viathe computing device indicative of the change in condition via the radiosystem. In another embodiment, the step or operation of alerting thedriver of the change condition(s) can further include or involve a stepor operation for alerting the driver of the change in the condition by atext message displayable through the computing device.

In another embodiment, the step or operation of monitoring thecondition(s) with respect to the vehicle can further involve monitoringof the condition with one or more cameras (e.g., a video camera,high-definition video camera, etc.). In some embodiments, one or more ofsuch cameras can be a 360 degree video camera. In other embodiments,monitoring the condition(s) with respect to the vehicle may also involvemonitoring condition(s) with one or more sensors, either by themselvesor in association with the aforementioned video camera(s). In anotherembodiment, the step or operation of monitoring the condition(s) withrespect to the vehicle can involve a step or operation of analyzingvideo data from the video camera(s) (e.g., an HD video camera, a 360degree video camera etc.) utilizing anomaly detection (e.g., an anomalydetection module, anomaly detection mechanism, etc.). In yet anotherembodiment, the step or operation of monitoring the condition(s) canfurther involve a step or operation for analyzing video data captured bythe video camera(s) utilizing machine learning. In still anotherembodiment, the step or operation of monitoring the condition(s) withrespect to the vehicle can further involve or include a step oroperation for analyzing video data obtained from the video camera(s)utilizing location data (e.g., GPS data, beacon data) from a locationmodule (e.g., GPS module, beacon module, etc.) associated with thecomputing device. The location data can be employed to cross-referencelocation identification information, for example, with respect to itemsor objects identified from the video data obtained from the videocamera(s).

In another embodiment, a method can be implemented for alerting avehicle driver via wireless communications, wherein such a methodincludes steps or operations for monitoring condition(s) external to avehicle while the vehicle is in operation and a driver of the vehicle islocated in a driver seat of the vehicle, detecting a change in thecondition(s); transmitting a signal wirelessly to a computing device,wherein the signal indicative of a change in the condition(s); andalerting the driver regarding the change in condition(s), aftertransmission of the signal to the computing device.

In some embodiments, the step of alerting the driver of the change inthe condition(s) after transmission of the signal to the computingdevice, can further involve a step or operation of providing or issuingan audible alert via a speaker associated with the computing device,wherein the audible alert is indicative of the change in condition(s).

In another embodiment, the step or operation of alerting the driver ofthe change in condition(s) after transmission of the signal to thecomputing device, can further include steps or operations forestablishing a wireless connection between the computing device and aradio system of the vehicle; and issuing or providing an audible alertfrom the computing device which is issued (e.g., played or broadcast)through the vehicle's radio system, wherein the audible alert isindicative of the change in condition. In such a situation the radiosystem is in communication with the computing device. The computingdevice may be a separate wireless mobile electronic device such as asmartphone, smartwatch or tablet computing device associated with thedriver and/or a passenger (or passengers) in the vehicle, or thecomputing device may be a computing system integrated with the vehicleand which communicates electronically with the radio system. In someembodiments, the “vehicle in operation” may involve the vehicle inmotion or the vehicle temporarily stopped (e.g., at an intersection,dropping or picking up a passenger, etc.). In another embodiment, thestep or operation of alerting the driver of the change in condition(s),after transmission of the signal to the computing device, can furtherinvolve a step or operation of providing an audible alert via a speakerassociated with the computing device, the audible alert indicative ofthe change in condition(s).

In yet another embodiment, the step or operation of alerting the driverof the change in the condition(s) after transmission of the signal tothe computing device, can further involve steps or logical operationsfor establishing a wireless connection between the computing device anda radio system of the vehicle; and providing an audible alert from thecomputing device indicative of the change in the condition(s) via theradio system. In still another embodiment, the step or operation ofmonitoring a condition external to the vehicle while the vehicle is inoperation and the driver of the vehicle is located in the driver seat ofthe vehicle, can further include a step or operation o monitoring thecondition(s) with a camera (e.g., video camera, HD camera, 360 degreevideo camera, etc.) that communicates with the computing device. In someembodiments, the video camera may be integrated with the computingdevice (e.g., a tablet computing device video camera, smartphone videocamera, etc.), while in other embodiments, the camera may be astandalone camera positioned within the vehicle to monitor thecondition(s) and which communicates via a wireless connection with thecomputing device (e.g., tablet computing device, smartphone, smartwatch,integrated in-vehicle computing system, etc.).

In still another embodiment, a step or operation can be provided fortracking and recording in a memory of a computing system (e.g., aserver, an in-vehicle computing system, a tablet computing device, asmartphone, a smartwatch, laptop computer, desktop computer, etc.), dataindicative of the number of times the driver is alerted to changes inconditions. In another embodiment, steps or operations can beimplemented for periodically retrieving such tracked and recorded datafrom the memory, and transmitting the data wirelessly from the computingsystem to a central repository (e.g., a server) for further storage andanalysis. Note that in some embodiments, the condition(s) external tothe vehicle may be a stop light condition (e.g., traffic light) and thechange in the condition(s) involves a change from one stop light colorto another stop light color (e.g. red to green, green to red, green toyellow, etc.).

In another embodiment the step or operation of transmitting the signalwirelessly to the computing device, wherein the signal is indicative ofthe change in condition(s), can further involve a step or operation fortransmitting the signal wirelessly through a PAN (Personal AreaNetwork). In some embodiments, the PAN may be a network enabled forBluetooth wireless communications, induction wireless communications,infrared wireless communications, ultra-wideband wireless communicationsand/or ZigBee wireless communications. In some embodiments, the wirelessconnection between the computing device and the radio system can beestablished via Secure Simple Pairing (SSP).

In another embodiment, the step or operation of detecting the change inthe condition(s) can involve utilizing anomaly detection or machinelearning approaches for detecting the change in the condition(s).

In yet another embodiment, steps or operations can be provided fordetermining if the vehicle is no longer in operation, monitoring thecondition(s) within the vehicle, in response to determining that thevehicle is no longer in operation, determining if the condition(s)within the vehicle comprises an anomalous condition; and wirelesslytransmitting an alert to a computing device associated with the user,the alert indicative of the anomalous condition, if is determined thatthe condition(s) comprises the anomalous condition (i.e., if it isdetermined that the condition or conditions are anomalous). Note that insome embodiments, the alert can be wirelessly transmitted as a textmessage to the computing device via a wireless network, wherein the textmessage is displayed via a display screen associated with the computingdevice.

In another embodiment, a system for alerting a vehicle driver viawireless communications, can be implemented. Such a system can include,for example, a video camera (i.e., one or more video cameras), one ormore processors which communicate with and process video data capturedby the video camera, and a computer-usable medium embodying computerprogram code, wherein the computer-usable medium capable ofcommunicating with the processor(s). The computer program code caninclude instructions executable by the processor(s) and configured, forexample, for: monitoring condition(s) with respect to the vehicle withthe video camera, detecting a change in the condition(s) monitored viathe video camera, transmitting a signal wirelessly to a computingdevice, the signal indicative of the change in condition(s), andalerting the driver of the change in condition(s) in response totransmitting the signal to the computing device. As indicatedpreviously, the computing device may be, for example, mobile electronicdevice such as a smartphone, tablet computing device, laptop computerand so on. In some embodiments, the computing device may be anin-vehicle computing system that communicates wirelessly with othercomputing devices such as the driver's and/or a passenger's smartphone,tablet computing device, etc.

In some embodiments, the instructions for alerting the driver of thechange condition(S) can be further configured for providing an audiblealert via a speaker associated with the computing device, wherein theaudible alert is indicative of the change condition(s). In yet anotherembodiment, the instructions for alerting the driver of the change inthe condition(s) can be further configured for establishing a wirelessconnection between the computing device and a radio system of thevehicle; and providing an audible alert from the computing deviceindicative of the change in the condition(s) via the radio system. Inanother embodiment, the instructions for alerting the driver of thechange in the condition(s) can be further configured for alerting thedriver of the change in the condition(s) by a text message displayablethrough the computing device and/or played or broadcast as a voice alertthrough the computing device. As indicated previously, the camera(s) maybe, for example, a video camera, HD video camera, 360 degree videocamera, and so on. In some embodiments, one or more sensors (e.g.,temperature sensor, pressure sensor, velocity sensor, accelerationsensor, vehicle heading sensor, etc.) may be employed for use inmonitoring the conditions external to or within the vehicle.

In some embodiments, the instructions for monitoring the condition(s)can be further configured for analyzing the video data captured from thevideo camera utilizing anomaly detection or machine learning techniques.In still other embodiments, the instructions for monitoring thecondition(s) can be configured for analyzing the video data capturedfrom the 360 degree video camera utilizing, for example, location datafrom a location or locating module (e.g., beacon module, GPS module,etc.) associated with the computing device and in association with thecaptured video data.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying figures, in which like reference numerals refer toidentical or functionally-similar elements throughout the separate viewsand which are incorporated in and form a part of the specification,further illustrate the present invention and, together with the detaileddescription of the invention, serve to explain the principles of thepresent invention.

FIG. 1 illustrates a schematic diagram of driver alerting system, inaccordance with a preferred embodiment;

FIG. 2(a) illustrates an example computing device constituting asmartphone or tablet computing device, which may be adapted for use inaccordance with one embodiment;

FIG. 2(b) illustrates an example camera, which may be located in avehicle for monitoring conditions external to the vehicle, in accordancewith another embodiment;

FIG. 3 illustrates a method for alerting a vehicle driver via wirelesscommunications, in accordance with an embodiment;

FIG. 4 illustrates a method for alerting a vehicle driver, in accordancewith an alternative embodiment;

FIG. 5 illustrates a method 50 for tracking driver activity, inaccordance with an alternative embodiment;

FIG. 6 illustrates a method 60 for monitoring conditions with respect toa vehicle, in accordance with an alternative embodiment;

FIG. 7 illustrates a schematic diagram of a system for alerting avehicle driver via wireless communications, in accordance with analternative embodiment;

FIG. 8 illustrates a schematic diagram of a system for alerting avehicle driver via wireless communications, in accordance with analternative embodiment;

FIG. 9 illustrates a schematic diagram of a system for alerting avehicle driver via wireless communications, in accordance with analternative embodiment;

FIG. 10 illustrates a method for alerting a vehicle driver of a changein traffic light conditions via wireless communications, in accordancewith an embodiment;

FIG. 11 illustrates a schematic view of a computer system, in accordancewith an embodiment;

FIG. 12 illustrates a schematic view of a software system including amodule(s), an operating system, and a user interface, in accordance withan embodiment; and

FIG. 13 illustrates a schematic diagram of driver alerting system, inaccordance with an alternative embodiment.

DETAILED DESCRIPTION

The particular values and configurations discussed in these non-limitingexamples can be varied and are cited merely to illustrate one or moreembodiments and are not intended to limit the scope thereof.

FIG. 1 illustrates a schematic diagram of a system 10 for alerting avehicle driver via wireless communications, in accordance with apreferred embodiment. System 10 monitors one or more conditions withrespect to a vehicle, preferably (but not necessarily) while the vehicleis in operation and the driver of the vehicle is located in thevehicle's driver's seat. The conditions may be external to the vehicleor internal depending upon the application or need. In the example shownin FIG. 1, the change in condition is depicted a change in the color ofa traffic light 22. A camera 19 (e.g., a video camera) is shown in FIG.1 as located on a dashboard with respect to a front vehicle windshield18. For example, the traffic light 22 may change from red to green andan alert 24 is issued alerting the driver of the change in color when achange in the condition (i.e., change in color, in this case) isdetected. The monitoring activity (e.g., monitoring the traffic light22) is indicated in FIG. 1 by dashed arrow 17. As will be discussed ingreater detail herein, such monitoring activity can be facilitated by athe camera 19 and other components and modules.

The alert 24 may be transmitted wirelessly to, for example, the user'smobile electronic wireless computing device 21 (e.g., a smartphone,tablet computing device, smartwatch, wearable device, etc.) or to adevice or system integrated with the vehicle. Note that electronicdevices such as smartphones, smartwatches, personal digital assistants(PDAs) mobile telephones, tablet devices, laptops and other Internetconnectivity devices (“mobile stations”) provide users with mobileaccess to various information resources. Such mobile stations generallyoperate via a cellular 3G or 4G broadband data communication standardand/or a WIFI network connection to a local area network. In FIG. 1 thecomputing device is shown as MD (Mobile Device) 21.

In the example of FIG. 1, the monitoring activity (e.g., monitoring thetraffic light or other conditions) can be facilitated by one or morecameras such as, for example, a camera 19 which may be located on thedashboard of the vehicle with a view through the vehicle windshield 18.In such a situation, the camera 19 may be, for example, a dashcam. Adashboard is a preferable location for camera 19, but it should beappreciated that the camera 19 may be located in other positions withinthe vehicle. In the example depicted in FIG. 1, only a single camera 19is shown. In other embodiments, as will be explained in greater detailherein, more than one camera may be implemented in the context of avehicle. In the FIG. 1 scenario, camera 19 is shown monitoring theexternal condition (e.g., traffic light 22 changing color) through thewindshield 18.

FIG. 2(a) illustrates an example computing device 21, which is shown asa smartphone implementation. It can be appreciated, however, that thecomputing device 21 can be implemented as other types of wireless mobiledevices, such as a tablet computing device, and wearable devices such asa smartwatch.

A non-limiting example of a wearable device such as a smartwatch, whichcan be utilized as computing device 21 is depicted in U.S. Pat. No.8,854,925 entitled “Smart Watch and Control Method for the Same,” whichissued on Oct. 7, 2014 and is incorporated herein by reference. Anothernon-limiting example of a smartwatch that can be adapted for use ascomputing device 21 is disclosed in U.S. Pat. No. 8,279,716 entitled“Smart-Watch Including Flip-Up Display,” which issued on Oct. 2, 2012and is incorporated herein by reference. Note that the terms “smartwatch” and “smartwatch” and “smart-watch) can be utilizedinterchangeably to refer to the same type of device.

Another example of a wearable device that can be implemented ascomputing device 21 is an OHMD (Optical Head-Mounted Display) that canbe equipped with a video camera. OHMD is a wearable display that has thecapability of reflecting projected images as well as allowing the userto see through it that is augmented reality.

The computing device 21 can incorporate a video camera 19. In someembodiments, the example computing device 21 with camera 19 may beimplemented in the context of, for example, a smartphone or tabletcomputing device located or mounted on the vehicle dashboard orelsewhere within the vehicle (or located on the vehicle external to thepassenger compartment). The alert 24 may be broadcast as an audio alertor text alert message through the computing device 21. In someembodiments, the alert can be transmitted in the context of a voicealert, which is discussed further herein.

In another embodiment, the camera 19 may be implemented as a standalonecamera that communicates wirelessly with the computing device 21 viawireless communications as described in greater detail herein. FIG. 2(b)illustrates an example camera 19, which may be located in an optimallocation with the vehicle for monitoring conditions external to thevehicle, in accordance with another embodiment. In the FIG. 2(b)implementation, the camera 19 may communicate wirelessly with thecomputing device 21. One non-limiting example of a video camera whichmay adapted for use as, for example, camera 19 shown in FIG. 2(b) isdisclosed in U.S. Patent Application Publication No. 20140047143entitled “Wireless video camera and connection methods including a USBemulation,” which issued on Feb. 13, 2014 and is incorporated herein byreference in its entirety.

Camera 19 may also be implemented as, for example, a so-called dashcamor dash cam. A dashcam (dashboard camera) is an onboard camera thatattaches to the vehicle's interior windscreen by either a suppliedsuction cup mount or an adhesive-tape mount. It can also be positionedon top of the dashboard or attached to the rear-view mirror with aspecial mount. It continuously records the road ahead while the vehicleis in motion. Various types of dashcam can be implemented as camera 19,ranging from basic video cameras to those capable of recordingparameters such as date/time, speed, G-forces and location. In someembodiments, camera 19 may be implemented as a wearable video camerathat monitors conditions external to the vehicle or within the vehicle.Such a video camera may be, for example, a lapel camera worn by a thevehicle driver and/or a passenger.

FIG. 3 illustrates a method 20 for alerting a vehicle driver viawireless communications, in accordance with an embodiment. As indicatedat block 24, the process begins. Thereafter, as shown at block 26, astep or logical operation can be implemented for monitoring one or moreconditions external to the vehicle and optionally while the vehicle isin operation and the driver of the vehicle is located in a driver seatof the vehicle.

Note that such a monitoring step or logical operation may involvemonitoring the condition with a camera that communicates with thecomputing device. The camera may be integrated the computing device(e.g., a Smartphone or tablet computer). In other embodiments, such acamera may be a standalone camera positioned within the vehicle tomonitor the condition and the camera may also communicate via a wirelessconnection (e.g., Bluetooth or other wireless communication as discussedin greater detail herein) with the computing device.

Monitoring can involve the use of object recognition or other videoimage recognition techniques and systems. For example, in one embodimenta traffic recognition approach can be utilized as part of the videomonitoring operation. One example of a traffic object recognitionapproach that can be adapted for use in accordance with an embodiment isdisclosed in U.S. Patent Application Publication No. 2011/0184895entitled “Traffic Object Recognition System, Method for Recognizing aTraffic Object, and Method for Setting up a Traffic Object RecognitionSystem,” which published to Janssen on Jul. 28, 2011 and is incorporatedherein by reference in its entirety. Another object recognition approachthat can be adapted for use in accordance with an alternative embodimentis disclosed in U.S. Pat. No. 8,447,139 entitled “Object recognitionusing Haar features and histograms of oriented gradients,” which issuedon May 21, 2013 and is incorporate herein by reference in its entirety.

Next, as illustrated at decision block 28, a test can be performed todetermine if a change has been detected the monitored conditions (orconditions). If a change is detected, then as disclosed at block 30, astep or logical operation can be implemented for transmitting a signalwirelessly to a computing device, wherein such a signal indicative ofthe change in the condition(s) monitored. Thereafter, as shown at block32, a step or logical operation can be implemented to alert the driverof the change in the condition after transmission of the signal to thecomputing device.

It can be appreciated that the vehicle in operation may be, for example,temporarily stopped (e.g., at an intersection/stop light, a parking lot,in traffic, etc.) or in motion. In some implementations, the computingdevice that receives and plays the alert (e.g., an audio signal or voiceannouncement) may be, for example, a smartphone or a tablet computingdevice. In other embodiments, the computing device may be integratedwith the vehicle as part of an in-vehicle system that provides alertsand other information (e.g., GPS information) to the vehicle'soccupants. Such a system typically includes a dashboard display. Oneexample of a non-limiting in-vehicle system that can be adapted for usein accordance with an alternative embodiment is disclosed in US PatentApplication Publication No. 20110034128 entitled “Mobile CommunicationDevice Linked to In-Vehicle System,” which published on Feb. 10, 2011and is incorporated herein by reference in its entirety. Yet anotherexample of a non-limiting in-vehicle system that can be adapted for usein accordance with an alternative embodiment is disclosed in U.S. Pat.No. 8,417,211 entitled “In-Vehicle System (IVS) Control of EmergencyData Communications,” which issued on Apr. 9, 2013 and is incorporatedherein by reference in its entirety.

It can also be appreciated that in the context of a tablet or smartphoneimplementation, the computing device may not necessarily belong to thevehicle driver but may, for example, be a computing device (e.g., handheld wireless electronic device, smartphone, tablet, etc.) belonging topassengers.

FIG. 4 illustrates a method 40 for alerting a vehicle driver, inaccordance with an alternative embodiment. In some embodiments, the stepor logical operation of alerting the driver of the change in conditionafter transmission of the signal to computing device, can furtherinvolve providing an audible alert via a speaker associated with thecomputing device, wherein the audible alert indicative of the change inthe monitored condition. Such an alerting operation may involveestablishing a wireless connection between the computing device and aradio system of the vehicle as shown at block 42, and providing theaudible alert from the computing device via the radio system, asindicated at block 44.

FIG. 5 illustrates a method 50 for tracking driver activity, inaccordance with an alternative embodiment. It can be appreciated that insome embodiments, the disclosed monitoring method/system can be modifiedto monitor not just external conditions or activities, but the activityof the driver itself in order to track, for example, driverinattentiveness. As depicted at block 52, a step or logical operationcan be implemented for tracking and recording in a memory of a computingsystem data indicative of the number of times the driver is alerted to achange a condition. Then, as shown at block 54, a step or logicaloperation can be implemented to periodically retrieve such data from thememory, and as shown at block 56, transmit such data wirelessly from thecomputing system to a central repository for further storage andanalysis. Such data may be useful, for example, for insurance companies.

FIG. 6 illustrates a method 60 for monitoring conditions with respect toa vehicle, in accordance with an alternative embodiment. As indicated atblock 62, the process is initiated. Thereafter, as depicted at decisionblock 64, a test can be performed to determine if the vehicle is inoperation. If it is determined that the vehicle is in operation, thenthe process ends, as shown at block 72. If it is determined that thevehicle is no longer in operation (e.g., the key is removed from theignition), then as indicated at block 66, conditions within the vehicleare monitored. For example, a camera and/or sensors may monitorconditions within the vehicle (or external to the vehicle).

As depicted at block 68, a test is performed to determine if conditionsanomalous. Anomalous conditions may include one of a variety of possibleconditions. For example, an anomalous condition may be a change intemperature in the vehicle. Another anomalous condition may be, forexample, the presence of someone in the vehicle who normally would notstill be in the vehicle after the car is turned off or, for example, thevehicle doors are closed and/or locked. If such an anomalous conditionis detected, then as indicated at block 70, an alert may be wirelesslytransmitted to a computing device associated with a user (e.g., thevehicle driver, a passenger, etc.) indicating such an anomalouscondition. The process can then terminate, as depicted at block 72. Notethat in some embodiments, such an alert may be wirelessly transmitted asa text message to the computing device via a wireless network. Such awireless network can be, for example, a cellular telephone networkand/or a WiFi network.

A text message alert can be implemented via for example, Short MessageService (SMS), SkyMail, Short Mail, Instant Messaging (IM), chat, MobileInstant Messaging (MiM), Multimedia Messaging Service (MMS), and othermessaging services. Text messaging is supported by computer devices suchas laptop computers, desktop computers, handheld computers, and wirelessdevices such as cellular telephones, Wireless Local Area Network (WLAN)terminals, Wireless Wide Area Network (WWAN) terminals, and WirelessPersonal Area Network (WPAN) terminals, for example.

Typically, a text message server serves as an intermediate device forreceiving a text message from a source device, storing the text message,and forwarding the text message to a recipient device, e.g., a firstcell phone as a source device and a second cell phone as a recipientdevice. While some text message service providers charge for textmessage support, e.g., cellular networks, other text message serviceproviders support text messaging without charge. Various protocols suchas SS7, GSM MAP, or TCP/IP, for example, may be employed to support textmessaging.

In some embodiments, the alert regarding a change in condition can beimplemented in the context of a notification service. In one example,the text message may be sent as a push notification across a cellular orwireless communication network to the computing device. Certain textmessaging protocols may be used, such as, mobile short message service(SMS), multimedia message service (MMS), and instant messaging (IM), orany other related text application. The communication medium may includetransferring information over communication links, such as wirelessnetworks (e.g., GSM, CDMA, 3G, 4G, etc.), wireline networks (e.g.,landline telephony), Internet, satellite/cable networks, or, any otherdata medium using standard communication protocols.

An example of a notification service that can be adapted for use with analternative embodiment is disclosed in U.S. Pat. No. 8,751,602 entitled“Method and Apparatus of Providing Notification Services to SmartphoneDevices,” which issued on Jun. 10, 2014 and is incorporated herein byreference in its entirety. Another non-limiting example of a system thatcan be adapted for use in accordance with an alternative embodiment fordelivery of an alert regarding a change in condition is disclosed inU.S. Pat. No. 8,265,938 entitled “Voice Alert Methods, Systems andProcessor-Readable Media,” which issued on Sep. 11, 2012 and isincorporated herein by reference in its entirety.

An example of a situation where the method 60 would be useful is thecase of a child accidentally being left in a vehicle during hot weather.A camera in the vehicle operating via battery power or residualelectricity from the vehicle electrical system may detect an anomalysuch as the child in a rear car seat. The anomaly in this case would bethe presence (e.g., detection of the child moving, turning his or herhead, moving his or her arms, legs, etc.) of a child in a car seat,wherein a child would not normally be in the car seat after the car isno longer in operation and/or after the doors are closed/locked, and/orafter a particular amount of time (e.g., 5 minutes, 10 minutes, etc.).Note that a cellular network or cellular link or service such as OnStarcan be utilized for sending out an alert (e.g., text message, audioalert) etc. to let them know that a child may have been left in a carseat.

Audio sensors may also be employed to detect, for example, the sound ofa crying child. A temperature sensor could also be utilized to detect arise in temperature to an unsafe level for humans and when thattemperature threshold is met, the alert is transmitted wirelessly to theuser's hand held device (e.g., smartphone, tablet, smartwatch or otherwearable device, etc.). Such an approach could thus be utilized toprevent tragic and unnecessary deaths in automobiles due to heatstroke.

Note that the step or logical operation of anomaly detection or outlierdetection shown in block 68 can involve the identification of items,events or observations which may not conform to an expected pattern orother items in a dataset. Anomalies are also referred to as outliers,novelties, noise, deviations and exceptions. In the context of abuse andnetwork intrusion detection, “interesting” objects are often not rareobjects, but unexpected bursts in activity. This pattern does not adhereto the common statistical definition of an outlier as a rare object, andmany outlier detection methods (in particular unsupervised methods) willfail on such data, unless it has been aggregated appropriately. Instead,a cluster analysis algorithm may be able to detect the micro dustersformed by these patterns.

The anomaly detection operation shown at block 68 can preferably beimplemented by an anomaly detection mechanism based on a number ofpossible categories of anomaly detection including but not limited to,unsupervised anomaly detection, supervised anomaly detection,semi-supervised anomaly detection, etc. An unsupervised anomalydetection technique can be employed detect anomalies in an unlabeledtest data set under the assumption that the majority of the instances inthe data set are normal by looking for instances that seem to fit leastto the remainder of the data set. Alternatively, a supervised anomalydetection technique may be employed, which requires a data set that hasbeen labeled as “normal” and “abnormal” and involves training aclassifier (the key difference to many other statistical classificationproblems is the inherent unbalanced nature of outlier detection).Semi-supervised anomaly detection techniques may also be employed, whichconstruct a model representing normal behavior from a given normaltraining data set, and then testing the likelihood of a test instance tobe generated by the learnt model.

FIG. 7 illustrates a schematic diagram of a system 70 for alerting avehicle driver via wireless communications, in accordance with analternative embodiment. System 70 includes a module (or group ofmodules) 72 including a monitoring module 74, an alerting module 76, atracking module 78, and a recording module 80. Module 72 can communicatewirelessly with a computing device 84 (e.g., a driver/passengersmartphone, tablet computer, etc.), which in turn can communicatewirelessly with the vehicle radio system 86. One or more sensor(s) 83can communicate with module 72 and one or more camera(s) 82 maycommunicate with the module 72. The sensor(s) 83 and the camera(s) 82can communicate with module 72 via wireless or wired communications.Note that the camera (or cameras) 82 are similar or analogous to thecamera 19 discusses previously herein. The computing device 84 is alsoin some embodiments similar or analogous to the electronic wirelessdevice 21 discusses previously herein.

Examples of sensors that can be utilized to implement sensor(s) 83 aresensors such as temperature sensors, pressure sensors, velocity sensors,acceleration sensors, vehicle heading sensors, yaw-rate sensors, and soon. One example of a vehicle heading sensor approach that can be adaptedfor use as or with sensor(s) 83 in accordance with an alternativeembodiment, is disclosed in U.S. Pat. No. 7,957,897 entitled “GPS-basedin-vehicle sensor calibration algorithm,” which issued on Jun. 7, 2011and is incorporated herein by reference in its entirety. The GPS modulediscussed herein can be utilized in association with such sensors toprovide location or position data with respect to the vehicle and alsoprovide vehicle heading sensor data.

Note that in some embodiments, the computing device 84 can communicatewith the vehicle radio system via wireless communications establishedvia Secure Simple Pairing (SSP). The sensor(s) 83 and the camera(s) 82and the computing device 84 may also in some embodiments communicatewith module 72 via SSP. SSP, which requires less user interactionutilizes a one-time six-digit key displays at the time of pairing onboth the device and the car, replacing the PIN code. Once the userconfirms that the keys match, the two devices can be paired.

The monitoring module 74 can implement the monitoring steps oroperations discussed previously. For example, monitoring module 74 canmonitor traffic lights or other conditions (i.e., conditions external tothe vehicle or within the vehicle) facilitated by, for example,camera(s) 82 and/or sensor(s) 83. The monitoring module 74 can be, forexample, an anomaly detection mechanism that detects changes inconditions as discussed previously.

The alerting module 76 serves to alert the driver of the detected changein a condition. The alert (e.g., an audio alert, a text message, etc.)can be broadcast through, for example, the computing device 84 or thevehicle radio system 86 (assuming the vehicle radio system 86 is pairedwith the computing device 84). The tracking module 78 and the recordingmodule 80 function to respectively track and record in a memory of acomputing system (e.g., the computing device 84, an onboard computingsystem, etc.) data indicative of, for example, the number of times thedriver is alerted to changes in conditions. Such data can be retrievedfrom the computer memory and then transmitted to, for example, a centralrepository for further storage and analysis.

It can be appreciated that in some cases, the connections between thevarious components shown in FIG. 7 may be implemented via wirelesscommunications and/or wired connections. For example, the module 72 canbe stored and retrieved from a computer memory which may be, forexample, a memory of computing device 84, a memory of an integratedin-vehicle computing system, and/or a remote computing device or systemsuch as a remote server. Wireless communications can occur through awireless network such as an in-car PAN (Personal Area Network) includingBluetooth communications, or other communications means such as, forexample, a cellular network.

FIG. 8 illustrates a schematic diagram of a system 70 for alerting avehicle driver via wireless communications, in accordance with analternative embodiment. System 70 shown in FIG. 8 is an alternativeversion of system 70 depicted in FIG. 7. The embodiment shown in FIG. 8includes similar or identical components to the FIG. 7 embodiment withsome slight variations. For example, FIG. depicts a wireless networkthrough which the various components can communicate. In one embodiment,for example, the wireless network 85 may implement an in vehicleBluetooth wireless communications system. The signal containing dataindicative of the alert can be wirelessly transmitted to the computingdevice 84 (assuming the computing device 84 in this case is, forexample, a Bluetooth enabled smartphone or tablet computing device). Insome embodiments, the computing device 84 (assuming a Bluetooth enableddevice) may be associated with one or more Bluetooth automotive speakerswherein the signal is processed via the computing device 84 as an audiosignal that can be played through the Bluetooth automotive speaker.

The wireless network 85 may be implemented as a PAN (Bluetooth orotherwise), and the signal transmitted through the PAN. It should beappreciated that wireless network 85 may be implemented not just viaBluetooth communications, but through one of a number of possiblealternative PAN wireless technologies. For example, in one embodimentwireless network 85 may be implemented as a PAN based on inductionwireless technology, which uses magnetic induction rather than radio forclose-range communications. In radio, both electric and magnetic fieldsmake up the signal, while in induction wireless, only the magnetic fieldis transmitted. The transmitter in this context is a radiating coil thatis more like the primary winding of a transformer than an antenna. A PANbased on an induction wireless approach has about a 3-m range. A typicalunit transmits up to 204.8-kbit/s data rates via GMSK modulation on 11.5MHz. Key benefits of induction wireless technologies are extremely lowpower consumption, low cost, and the inherent security that accompaniesshort range.

Another implementation of wireless network 85 can involve the use ofinfrared wireless communications. Such a PAN technology can be employedfor use over short distances. The IrDA infrared (IR) standard appearedduring the early 1990s, and can be utilized to implement wirelessnetwork 85 as a PAN network. IrDA initially offered a 115.2-kbit/s datarate over a range of up to 1 m. A 4-Mbit/s version was soon developedand has been widely incorporated in laptops and PDAs for printerconnections and short-range PANs. A 16-Mbit/s version is available too

The problem with IrDA is not just its very short range, but also itsneed for a line-of-sight (LOS) connection. Of course, Bluetooth does notneed LOS, and it can blast through walls. A more recent IR developmentis IrGate, which was produced by Infra-Com Technologies. This new IRdevelopment uses arrays of high-powered IR LEDs to emit coded basebandIR in all directions. Then, it relies on an array of photodetectors andsuper-sensitive receivers to pick up the diffused IR within thenetworking space. Thus, the LOS problem is mitigated, and a data rate ofup to 10 Mbits/s is possible.

Still another wireless technology for implementing wireless network 85in the context of, for example, an in-vehicle PAN is UWB (UltraWideband), which transmits data by way of baseband pulses applieddirectly to the antenna. The narrow pulses (less than 1 ns) create anextremely broad bandwidth signal. The pulses are modulated by pulseposition modulation (PPM) or binary phase-shift keying (BPSK). The FCCpermits UWB in the 3.1- to 10.6-GHz band. Its primary application todate has been short-range, high-resolution radar and imaging systemsthat penetrate walls, the ground, and the body. In addition, this newtechnology is useful for short-range LANs or PANs that require very highdata rates (over 100 Mbits/s).

Still another wireless technology for implementing wireless network 85in the context of, for example, an in-vehicle PAN is ZigBee, which is asimpler, slower lower-power, lower-cost cousin of Bluetooth, ZigBee.ZigBee is supported by a mix of companies that are targeting theconsumer and industrial markets. It may be a better fit with games,consumer electronic equipment, and home-automation applications thanBluetooth. Short-range industrial telemetry and remote control are othertarget applications. It can be appreciated, however, that wirelessnetwork 85 can be implemented as a ZigBeen PAN.

Previously referred to as RF-Lite, ZigBee is similar to Bluetoothbecause it uses the 2.4-GHz band with frequency-hopping spread-spectrumwith 25 hops spaced every 4 MHz. The basic data rate is 250 kbits/s, buta slower 28-kbit rate is useful for extended range and greaterreliability. With a 20-dBm power level, ZigBee can achieve a range of upto 134 meters at 28 kbits/s. It additionally allows for networking of upto 254 nodes.

Note that in some embodiments, whether that of FIG. 7 or FIG. 8 or otherimplementations, camera(s) 82 may be implemented as a 360 degree camerawhich can be employed for use in monitoring not only conditions withinthe vehicle but external to vehicle. In a preferred embodiment, camera(s) 82 and/or camera 19 discussed earlier can be implemented as an HD(High Definition) 360 degree camera that provides quality video datathat can be monitored via, for example, anomaly detection, machinelearning and other techniques.

An example of a 360 degree camera that can be adapted for use with oneor more embodiments is the Giroptic 360cam by Gripoptic. Such a deviceincludes three 185-degree fish-eye cameras, allowing it to capture 360degrees of HD video and photos (including time-lapse and HDR). TheGiroptic 360cam captures audio as well as video, and can record 3D soundfrom three microphones. Media can be saved onto a microSD card, which isthen loaded onto a computer via a micro USB port on the unit's base, orvia Wi-Fi. It can be appreciated that such a device (or other 360 degreevideo cameras) can be modified to communicate via other types ofwireless communications, such as Bluetooth communications, cellular, andso forth as discussed herein. Note that reference herein to the Giropticvideo camera is for illustrative purposes only and is not considered alimiting feature of the disclosed embodiments.

FIG. 9 illustrates a schematic diagram of a system 70 for alerting avehicle driver via wireless communications, in accordance with analternative embodiment. In the alternative embodiment of system 70 shownin FIG. 7, the monitoring module 72 can utilize an AD (AnomalyDetection) mechanism or module as discussed previously and/or ML(Machine Learning) and/or GPS (Global Positioning Satellite) modules.

ML techniques can be employed in the context of, for example, analgorithm that operates by building a model from example inputs and usedto make predictions or decisions, rather than following strictly staticprogram instructions. ML can be used to construct a model or rule set topredict a result based on values with respect to a number of features. Aseries of input patterns can be provided to an algorithm along with adesired output (e.g., the label) and the algorithm then learns how toclassify the patterns by outing a desired label. In supervised learning(e.g., Kemal-based support vector machine (SVM) algorithm), a humanoperator must provide the labels during a teaching phase. Alternatively,unsupervised clustering is a process of assigning labels to the inputpattems without the use of the human operator. Such unsupervised methodsgenerally function through a statistical analysis of the input data bydetermining an Eigen value vector of a covariance matrix.

One non-limiting ML technique that can be adapted for use in accordancewith an embodiment is AHaH (Anti-Hebbian and Hebbian) learning, whichcan be employed for feature extraction. One example of an AHaH MLapproach is disclosed in U.S. Pat. No. 8,918,353 entitled “Methods andSystems for Feature Extraction,” which issued on Dec. 23, 2014 and isincorporated herein by reference in its entirety. Another non-limitingML technique that can be adapted in accordance with another embodimentis disclosed in U.S. Pat. No. 8,429,103 entitled “Native MachineLearning Service for User Adaptation on a Mobile Platform,” which issuedon Apr. 23, 2013 and is incorporated herein by reference in itsentirety. It can be appreciated that such ML approaches are referred tofor illustrative purposes only and are not considered limiting featuresof the disclosed embodiments.

In the context of the embodiment shown in FIG. 9, ML can be combinedwith the AD mechanism to recognize patterns in, for example, video datacaptured by video camera (s) 19, 82, etc. to detect changes inconditions external to the vehicle or within the vehicle. Location datamay also be employed to determine the location of the vehicle withrespect to conditions being monitored external or within the vehicle.Location data may include, for example, GPS data and/or other locationdata, such as, beacon data (e.g., “iBeacon” data, etc.). Note that FIG.9 illustrates the use of a GPS module and GPS data (i.e., see “GPS” inFIG. 9). It can be appreciated that other types of location data mayalso be employed such as beacon data (e.g., see FIG. 13).

For example, in some embodiments the monitoring operation of monitoringmodule 74 can involve estimating the distance to a particular point orlocation near the vehicle and providing a notification/alert via thealerting module 75 in the form of an audio alert, text message etc. MLand/or AD modules or mechanisms can be employed to detect changes inconditions with respect to particular geographic locations. For example,the GPS data may be utilized to determine that the vehicle is rapidlyapproaching particular crosswalk or intersection and an alert issued tolet the driver know that he or she is approaching this particularcrosswalk or intersection, while the ML and/or AD modules or techniquescan be employed to determine if someone is in the middle of thecrosswalk/intersection.

Possible alerts or conditions to be monitored and alerted can be, forexample, “approaching a red light,” “changing lanes,” “approaching amedian,” “15 feet to a median,” “10 feet to a median,” “at median,” etc.The camera(s) 19, 82 and so forth and the monitoring module 75 can lookfor conditions such as medians, red lights, yellow lights etc. anddetermines how far away these things are from the vehicle.

FIG. 10 illustrates a high level flow chart of operations depictinglogical operational steps of a method 90 for alerting a vehicle driverof a change in traffic light conditions via wireless communications, inaccordance with an alternative embodiment. As indicated at block 92, theprocess can be initiated. Thereafter, as disclosed at block 94, a stepor logical operation can be implemented for detecting if the driver'svehicle/automobile is in motion. Thereafter, as shown at decision block96, a test can be performed to determine if the automobile is in motion.If so, then the monitoring process continues, as shown at block 94. Ifnot, then as indicated at block 98, a video camera (e.g., cameras 19,21, etc.) can monitor the vehicle's surroundings for traffic lightconditions. For example, a step or operation can be implemented tosearch for and identify a traffic light and its conditions (e.g., red,green or yellow lights).

Note that a non-limiting example of a camera that can be adapted for usein accordance with the operation shown as block 98 and in someimplementations for us as the camera 19 discussed earlier herein is acolor recognition camera. A non-limiting example of a color recognitioncamera is disclosed in U.S. Pat. No. 6,803,956 entitled “ColorRecognition Camera,” which issued on Oct. 12, 2004 and is incorporatedherein by reference in its entirety. Such an example color recognitioncamera includes a red-green-blue CCD-imaging device that provides ananalog RGB-video signal. A set of three analog-to-digital convertersconverts the analog RGB-video signal into a digital RGB-video signal. Adigital comparator tests the digital RGB-video signal pixel-by-pixel fora match against a color setpoint. If a match occurs, a pixel with aparticular color represented by the color setpoint has been recognizedand a “hit” is output. A pixel address counter provides a pixel addressoutput each time a “hit” is registered. The number of hits per videoframe are accumulated, and a color-match area magnitude value is outputfor each frame. Alternatively, neural networks can be used to indicatehits when a pixel in the video image comes close enough to the colorsetpoint value. Just how close can be “learned” by the neural network.

As indicated next at block 100, a step or logical operation can beimplemented to determine if the light is red, green or yellow. If it isdetermined, as shown at block 102, that the light is red, then thetraffic light is monitored to determine if there is change from togreen. Assuming that the light changes from red to green, an alert(e.g., audio) is then issued indicating the change from red to green, asdepicted at block 104. The process can then terminate, as shown at block106.

Note that another color recognition approach that can be adapted for usein accordance with an alternative embodiment and for monitoring a changein color (e.g., traffic light change from yellow to green, red to green,etc.) is disclosed in U.S. Pat. No. 8,139,852 entitled “Colorclassification method, color recognition method, color classificationapparatus, color recognition apparatus, color recognition system,computer program, and recording medium,” which issued on Mar. 12, 2012and is incorporated herein by reference in its entirety.

Note that in some embodiments, computer program code for carrying outoperations of the disclosed embodiments may be written in an objectoriented programming language (e.g., Java, C#, C++, etc.). Such computerprogram code, however, for carrying out operations of particularembodiments can also be written in conventional procedural programminglanguages, such as the “C” programming language or in a visuallyoriented programming environment, such as, for example, Visual Basic.

The program code may execute entirely on the user's computer, partly onthe user's computer, as a stand-alone software package, partly on theuser's computer and partly on a remote computer or entirely on theremote computer. In the latter scenario, the remote computer may beconnected to a user's computer through a local area network (LAN) or awide area network (WAN), wireless data network e.g., Wi-Fi, Wimax,802.xx, and cellular network or the connection may be made to anexternal computer via most third party supported networks (e.g., throughthe Internet via an Internet Service Provider).

The embodiments are described at least in part herein with reference toflowchart illustrations and/or block diagrams of methods, systems, andcomputer program products and data structures according to embodimentsof the invention. It will be understood that each block of theillustrations, and combinations of blocks, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general-purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe block or blocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function/act specified in the various block orblocks, flowcharts, and other architecture illustrated and describedherein.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the functions/acts specified inthe block or blocks.

FIGS. 11-12 are shown only as exemplary diagrams of data-processingenvironments in which embodiments may be implemented. It should beappreciated that FIGS. 11-12 are only exemplary and are not intended toassert or imply any limitation with regard to the environments in whichaspects or embodiments of the disclosed embodiments may be implemented.Many modifications to the depicted environments may be made withoutdeparting from the spirit and scope of the disclosed embodiments.

As illustrated in FIG. 11, some embodiments may be implemented in thecontext of a data-processing system 200 that can include, for example,one or more processors such as processor 141, a memory 142, a controller143, a peripheral USB (Universal Serial Bus) connection 147, a display146, an input device (e.g., a mouse, touch screen display, etc.), akeyboard, etc. Data-processing system 200 may be, for example, a clientcomputing device (e.g., a client PC, laptop, tablet computing device,smartphone, etc.) which can communicate with, for example, a server (notshown) and/or other devices (e.g., wireless and/or wiredcommunications).

As illustrated, the various components of data-processing system 200 cancommunicate electronically through a system bus 151 or similararchitecture. The system bus 151 may be, for example, a subsystem thattransfers data between, for example, computer components withindata-processing system 200 or to and from other data-processing devices,components, computers, etc. Data-processing system 200 may beimplemented as, for example, a server in a client-server based network(e.g., the Internet) or can be implemented in the context of a clientand a server (i.e., where aspects are practiced on the client and theserver). Data-processing system 200 may also be, for example, astandalone desktop computer, a laptop computer, a Smartphone, a padcomputing device and so on. In the case of a smartphone, it can beassumed that devices such as keyboard 144, input unit 145 and so onwould implemented in the context of a touch screen display or otherappropriate mobile input interface. The data-processing system 200 canalso include or communicate with an image capturing unit 132 (e.g., avideo camera such as discussed herein, etc.).

FIG. 12 illustrates a computer software system 250 for directing theoperation of the data-processing system 200. Software application 254,stored for example in memory 202, generally includes a kernel oroperating system 251 and a shell or interface 253. One or moreapplication programs, such as software application 254, may be “loaded”(i.e., transferred from, for example, mass storage 207 or other memorylocation into the memory 201) for execution by the data-processingsystem 200. The data-processing system 200 can receive user commands anddata through the interface 253; these inputs may then be acted upon bythe data-processing system 200 in accordance with instructions fromoperating system 251 and/or software application 254. The interface 253in some embodiments can serve to display results, whereupon a user 249may supply additional inputs or terminate a session. The softwareapplication 254 can include one or more modules such as module 252,which can, for example, implement instructions or operations such asthose described herein.

The following discussion is intended to provide a brief, generaldescription of suitable computing environments in which the system andmethod may be implemented. Although not required, the disclosedembodiments will be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a single computer. In most instances, a “module” constitutesa software application.

Generally, program modules include, but are not limited to, routines,subroutines, software applications, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types and instructions. Moreover, those skilled in the artwill appreciate that the disclosed method and system may be practicedwith other computer system configurations, such as, for example,hand-held devices, multi-processor systems, data networks,microprocessor-based or programmable consumer electronics, networkedPCs, minicomputers, mainframe computers, servers, and the like.

Note that the term module as utilized herein may refer to a collectionof routines and data structures that perform a particular task orimplements a particular abstract data type. Modules may be composed oftwo parts: an interface, which lists the constants, data types,variable, and routines that can be accessed by other modules orroutines; and an implementation, which is typically private (accessibleonly to that module) and which includes source code that actuallyimplements the routines in the module. The term module may also simplyrefer to an application, such as a computer program designed to assistin the performance of a specific task, such as word processing,accounting, inventory management, etc.

FIGS. 11-12 are thus intended as examples and not as architecturallimitations of disclosed embodiments. Additionally, such embodiments arenot limited to any particular application or computing or dataprocessing environment. Instead, those skilled in the art willappreciate that the disclosed approach may be advantageously applied toa variety of systems and application software. Moreover, the disclosedembodiments can be embodied on a variety of different computingplatforms, including, for example, Windows, Macintosh, UNIX, LINUX, andthe like.

FIG. 13 illustrates a schematic diagram of driver alerting system 150,in accordance with an alternative embodiment. Note that the system 150shown in FIG. 13 is similar to, for example, system 10 shown in FIG. 1.In the FIG. 13 implementation, however, a beacon 13 is shown asassociated with traffic light 22 and can be accessible by nearby mobileelectronic devices such as mobile device 21. The beacon 13 isimplemented as part of a positioning system. One example of beacon 13 isthe “iBeacon” device and associated system. The iBeacon is a trademarkof Apple Inc. for a positioning system that Apple Inc. has referred toas a new class of low-powered, low-cost transmitters that can notifynearby iOS devices of their presence. The technology enables asmartphone or other device to perform actions when in close proximity toan iBeacon or in this case beacon 13.

Thus, beacon 13 can assist the mobile computing device 21 in determiningits approximate location or context. With the assistance of beacon 13,software associated with mobile computing device 21 can approximatelyfind its relative location with respect to beacon 13 and hence withrespect to the traffic light 22 (assuming the beacon 13 is located at orproximate to the traffic light 22). The beacon 13 can communicate withdevice 21 using BLE (Bluetooth Low Energy) technology also referred toas “Bluetooth Smart”. The beacon 13 uses low energy proximity sensing totransmit a universally unique identifier picked up be a compatible “app”or operating system. The identifier can then be looked up over theInternet to determine the physical location of device 21 or trigger anaction on device 21 such as a push notification or tracking andrecording operations as discussed previously herein. One non-limitingexample of a beacon device and systems that can be adapted for use as orwith device 21 and beacon 13 and the methods/systems disclosed herein isdiscussed in U.S. Pat. No. 8,718,620 entitled “Personal Media Deviceswith Wireless Communication,” which issued on May 6, 2014 and isincorporated herein by reference.

Based on the foregoing, it can be appreciated that a number ofembodiments, preferred and alternative, are disclosed herein. Forexample, in one embodiment a method for alerting a vehicle driver viawireless communications can be implemented. Such a method can includethe steps or logical operations of, for example, monitoring one or moreconditions with respect to a vehicle; detecting a change in thecondition (or conditions); transmitting a signal wirelessly to acomputing device, the signal indicative of the change in thecondition(s); and alerting the driver of the change in the condition(s)in response to transmitting the signal to the computing device. In someembodiments, the computing device may be, for example, a wireless handheld electronic device such as a smartphone or tablet computing device(e.g., iPad, Android tablet, etc.). In other embodiments, the tablecomputing device may actually be integrated with the vehicle.

In some embodiments, the step or logical operation of alerting thedriver of the change in the condition(s) further comprises providing anaudible alert via a speaker associated with the computing device, theaudible alert indicative of the change in condition(s). In anotherembodiment, the step or logical operation of alerting the driver of thechange in the condition(s) can further include steps or logicaloperations for establishing a wireless connection between the computingdevice and a radio system of the vehicle; providing an audible alertfrom the computing device indicative of the change in the condition(s)via the radio system.

In yet another embodiment, the step or logical operation of alerting thedriver of the change in the condition(s) can further include the step orlogical operation of alerting the driver of the change in the conditionby a text message displayable through the computing device. In stillanother embodiment, monitoring the condition(s) with respect to avehicle can further involve monitoring such condition(s) with a camera.In some embodiments, such a camera may be for example, an HD videocamera, a 360 degree video camera, etc.

In other embodiments, the step or logical operation of monitoring thecondition(s) with respect to the vehicle can further include a step orlogical operation of monitoring the condition(s) with one or moresensors (e.g., temperature sensor, tire pressure sensor, etc.). In yetother embodiments, the step or logical operation of monitoring thecondition(s) can further involve a step or logical operation ofanalyzing video data from the 360 degree video camera utilizing anomalydetection. In still other embodiments, the step or logical operation ofmonitoring the condition(s) can further involve the step or logicaloperation of analyzing video data from the 360 degree video camerautilizing machine learning. In yet another embodiment the step orlogical operation of monitoring the condition(s) can further include astep or logical operation of monitoring the condition(s) by analyzingvideo data from the 360 degree video camera utilizing location data(e.g., GPS data, beacon data, etc.) from a location module (e.g., GPSmodule, iBeacon, etc.) associated with, for example, the computingdevice or the vehicle itself (e.g., an in-vehicle mounted GPS unit).

In another embodiment, a method for alerting a vehicle driver viawireless communications, can be implemented. Such a method can includethe steps or logical operations of monitoring one or more conditionsexternal to a vehicle while the vehicle is in operation and a driver ofthe vehicle is located in a driver seat of the vehicle; detecting achange in the condition(s); transmitting a signal wirelessly to acomputing device, the signal indicative of the change in thecondition(s); and alerting the driver of the change in the condition(s)after transmission of the signal to the computing device.

In some embodiments, the step or logical operation of alerting thedriver of the change in the condition(s) after transmission of thesignal to the computing device, can further include a step or logicaloperation of providing an audible alert via a speaker associated withthe computing device, the audible alert indicative of the change in theat least one condition. In still another embodiment, the step or logicaloperation of alerting the driver of the change in the condition(s) aftertransmission of the signal to the computing device, can further includethe steps or logical operation of establishing a wireless connectionbetween the computing device and a radio system of the vehicle; andproviding an audible alert from the computing device indicative of thechange in the condition(s) via the radio system. In some embodiments,the “vehicle in operation” can include at least one of: the vehicle inmotion, or the vehicle temporarily stopped (i.e., yet still inoperation, such as the vehicle temporarily stopped at an intersection).

In another embodiment, the step or logical operation of alerting thedriver of the change in the at least one condition after transmission ofthe signal to the computing device, can further include the step orlogical operation of providing an audible alert via a speaker associatedwith the computing device, the audible alert indicative of the change inthe at least one condition. In another embodiment, the step of logicaloperation of alerting the driver of the change in the at least onecondition after transmission of the signal to the computing device, canfurther include the steps or logical operations of establishing awireless connection between the computing device and a radio system ofthe vehicle; and providing an audible alert from the computing deviceindicative of the change in the at least one condition via the radiosystem.

In still another embodiment, the step or logical operation of monitoringa condition external to a vehicle while the vehicle is in operation anda driver of the vehicle is located in a driver seat of the vehicle, canfurther include the step or logical operation of monitoring the at leastone condition with a camera that communicates with the computing device.In some embodiments, the aforementioned camera may be integrated withthe computing device or can be a standalone camera positioned within thevehicle to monitor the at least one condition and wherein the standalonecamera communicates via a wireless connection with the computing device.In some cases more than one camera may be employed (e.g., both thestandalone camera and the camera integrated with the computing device).

In another embodiments, steps or logical operations can be provided fortracking and recording in a memory of a computing system data indicativeof a number of times the driver is alerted to the change in the at leastone condition. Additionally, steps or logical operations may be providedfor periodically retrieving the data from the memory; and transmittingthe data wirelessly from the computing system to a central repositoryfor further storage and analysis.

In some embodiments, the at least one condition external to the vehiclemay be, for example, a stop light condition and the change in the atleast one condition comprises a change from one stop light color toanother stop light color.

In another embodiment the step or logical operation of transmitting thesignal wirelessly to the computing device, the signal indicative of thechange in the at least one condition, can further include or involve astep or logical operation of transmitting the signal wirelessly througha PAN (Personal Area Network). In some embodiments, such a PAN can be anetwork enabled for example, for: Bluetooth wireless communications,induction wireless communications, infrared wireless communications,ultra-wideband wireless communications and ZigBee wirelesscommunications. In some embodiments, the wireless connection between thecomputing device and the radio system can be established via SecureSimple Pairing (SSP).

In other embodiments, the step or logical operation of detecting thechange in the at least one condition can further involve a step orlogical operation for utilizing an anomaly detection mechanism to detectthe change in the at least one condition.

In another embodiment, steps or logical operations can be provided fordetermining if the vehicle is no longer in operation; monitoring atleast one condition within the vehicle, in response to determining thatthe vehicle is no longer in operation; determining if the at least onecondition within the vehicle comprises an anomalous condition; andwirelessly transmitting an alert to a computing device associated withthe user, the alert indicative of the anomalous condition, if isdetermined that the at least one condition comprises the anomalouscondition. In yet another embodiment the alert can be wirelesslytransmitted as a text message to the computing device via a wirelessnetwork.

In another embodiment, a system for alerting a vehicle driver viawireless communications, can be implemented. Such a system can include,for example, a video camera (one or more video cameras), at least oneprocessor that communicates with and processes video data captured bythe video camera; and a computer-usable medium embodying computerprogram code, the computer-usable medium capable of communicating withthe at least one processor. The computer program code can includeinstructions executable by the at least one processor and configured forexample, for: monitoring at least one condition with respect to avehicle with the video camera; detecting a change in the at least onecondition monitored with the video camera; transmitting a signalwirelessly to a computing device, the signal indicative of the change inthe at least one condition; and alerting the driver of the change in theat least one condition in response to transmitting the signal to thecomputing device. As indicated previously, the computing device may be,for example, a smartphone, a tablet computing device or an-vehiclecomputing system (or a combination of both).

In some embodiments, the aforementioned instructions for alerting thedriver of the change in the at least one condition can be furtherconfigured for providing an audible alert via a speaker associated withthe computing device, the audible alert indicative of the change in theat least one condition. In some situations, the speaker may be, forexample, a speaker integrated with a smartphone, tablet computingdevice, etc. The speaker may also be an audio speaker associated with anin-vehicle system such as discussed herein.

In other embodiments, the aforementioned instructions for alerting thedriver of the change in the at least one condition can be furtherconfigured for establishing a wireless connection between the computingdevice and a radio system of the vehicle; and providing an audible alertfrom the computing device indicative of the change in the at least onecondition via the radio system.

In yet another embodiment, the instructions for alerting the driver ofthe change in the at least one condition can further includeinstructions configured for alerting the driver of the change in thecondition by a text message displayable through the computing device.

In some embodiments, such a camera may be a 360 degree video camera. Inother embodiments, one or more sensors can also be employed for use inmonitoring the condition(s). In another embodiment, the instructions formonitoring the at least one condition can include instructions foranalyzing the video data captured from the 360 degree video camerautilizing anomaly detection. In yet another embodiment, the instructionsfor monitoring the at least one condition can further includeinstructions for analyzing the video data captured from the 360 degreevideo camera utilizing machine learning. In still another embodiment,the instructions for monitoring the at least one condition can furtherinvolve or include instructions for analyzing the video data capturedfrom the 360 degree video camera utilizing location data from a location(e.g., GPS module, beacon, etc.) associated with the computing device.

The flowcharts and block diagrams in the different depicted embodimentsillustrate the architecture, functionality, and operation of somepossible implementations of apparatus, methods and computer programproducts. In this regard, each block in the flowchart or block diagramsmay represent a module, segment, or portion of code, which comprises oneor more executable instructions for implementing the specified functionor functions. In some alternative implementations, the function orfunctions noted in the block may occur out of the order noted in thefigures. For example, in some cases, two blocks shown in succession maybe executed substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved.

The invention can take the form of an entire hardware embodiment, anentire software embodiment, or an embodiment containing both hardwareand software elements. In a preferred embodiment, the invention isimplemented in software, which includes but is not limited to firmware,resident software, microcode, etc.

Furthermore, the invention can take the form of a computer programproduct accessible from a computer-usable or computer-readable mediumproviding program code for use by or in connection with a computer orany instruction execution system. For the purposes of this description,a computer-usable or computer readable medium can be any tangibleapparatus that can contain, store, communicate, propagate, or transportthe program for use by or in connection with the instruction executionsystem, apparatus, or device.

The medium can be an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system (or apparatus or device). Examples ofa computer-readable medium include a semiconductor or solid statememory, magnetic tape, a removable computer diskette, a random accessmemory (RAM), a read-only memory (ROM), a rigid magnetic disk and anoptical disk. Current examples of optical disks include compactdisk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) andDVD.

Some embodiments may be implemented in the context of a so-called “app”or software application. An “app” is a self-contained program or pieceof software designed to fulfill a particular purpose; an application,especially as downloaded by a user to a mobile device (e.g., smartphone,tablet computing device, etc.).

It will be appreciated that variations of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be desirablycombined into many other different systems or applications. It will alsobe appreciated, that various presently unforeseen or unanticipatedalternatives, modifications, variations or improvements therein may besubsequently made by those skilled in the art, which are also intendedto be encompassed by the following claims.

What is claimed is:
 1. A system for alerting a vehicle driver viawireless communications, said system comprising: at least one processor;and a non-transitory computer-usable medium embodying computer programcode, said computer-usable medium capable of communicating with said atleast one processor, said computer program code comprising instructionsexecutable by said at least one processor and configured for: monitoringat least one condition external to a vehicle while said vehicle is inoperation and a driver of said vehicle is located in a driver seat ofsaid vehicle and said vehicle is stopped; detecting a change in said atleast one condition external to said vehicle during said monitoring ofsaid at least one condition while said vehicle is stopped; transmittinga signal wirelessly to a computing device, said signal indicative ofsaid change in said at least one condition external to said vehicle; andalerting said driver of said change in said at least one conditionexternal to said vehicle after transmission of said signal to saidcomputing device.
 2. The system of claim 1 wherein said instructions foralerting said driver of said change in said at least one conditionexternal to said vehicle after transmission of said signal to saidcomputing device, further comprises instructions configure for:providing an audible alert via a speaker associated with said computingdevice, said audible alert indicative of said change in said at leastone condition external to said vehicle.
 3. The system of claim 1 whereinsaid instructions for alerting said driver of said change in said atleast one condition external to said vehicle after transmission of saidsignal to said computing device, further comprises instructionsconfigured for: establishing a wireless connection between saidcomputing device and a radio system of said vehicle; providing anaudible alert from said computing device indicative of said change insaid at least one condition external to said vehicle, wherein saidaudible alert is broadcast via said radio system, after saidestablishing said wireless connection between said computing device andsaid radio system of said vehicle.
 4. The system of claim 1 wherein saidat least one condition comprises at least one condition of a trafficlight and said change in said at least one condition external to saidvehicle comprises a change in said at least one condition of saidtraffic light.
 5. The system of claim 1 wherein said computing devicecomprises at least one of a smartphone and a tablet computing device. 6.The system of claim 1 wherein said at least one condition external tosaid vehicle comprises a stop light condition and said change in said atleast one condition comprises a change from one stop light color toanother stop light color.
 7. The system of claim 1 wherein instructionsfor alerting said driver of said change in said at least one conditionafter transmission of said signal to said computing device, furthercomprises instructions configured for: establishing a wirelessconnection between said computing device and a radio system of saidvehicle; providing an audible alert from said computing deviceindicative of said change in said at least one condition via said radiosystem.
 8. The system of claim 1 wherein said instructions formonitoring a condition external to a vehicle while said vehicle is inoperation and a driver of said vehicle is located in a driver seat ofsaid vehicle, further comprises instructions configured for: monitoringsaid at least one condition with a camera that communicates with saidcomputing device.
 9. The system of claim 1 wherein said instructions arefurther configured for: tracking and recording in a memory of acomputing system data indicative of a number of times said driver isalerted to said change in said at least one condition external to saidvehicle while said vehicle is stopped.
 10. The system of claim 9 whereinsaid instructions are further configured for: periodically retrievingsaid data from said memory; and transmitting said data wirelessly fromsaid computing system to a central repository for further storage andanalysis.
 11. A method for alerting a vehicle driver via wirelesscommunications, said method comprising: monitoring at least onecondition with respect to a vehicle with said video camera while saidvehicle is stopped; detecting a change in said at least one conditionduring said monitoring of said at least one condition with a videocamera wherein said change in said at least one condition is detected bysaid video camera while said vehicle is stopped; transmitting a signalwirelessly to a computing device, said signal indicative of said changein said at least one condition detected by said video camera; andalerting said driver of said change in said at least one condition inresponse to transmitting said signal to said computing device, whereinsaid alerting said driver of said change in said at least one condition,further comprises establishing a wireless connection between saidcomputing device and a radio system of said vehicle and providing anaudible alert from said computing device indicative of said change insaid at least one condition via said radio system; and wherein saidwireless connection comprises a PAN (Personal Area Network) and whereinsaid instructions for alerting said driver of said change in said atleast one condition further comprise instructions configured foralerting said driver of said change in said at least one condition by atext message displayable through said computing device andsimultaneously broadcast as a voice alert through said computing deviceand said radio system, said voice alert including said audible alert.12. The method of claim 11 wherein said computing device comprises atleast one of a smartphone and a tablet computing device.
 13. The methodof claim 11 wherein said instructions for alerting said driver of saidchange in said at least one condition are further configured forproviding an audible alert via a speaker associated with said computingdevice, said audible alert indicative of said change in said at leastone condition while said vehicle is stopped.
 14. The method of claim 11wherein said video camera comprises a 360 degree video camera andwherein said at least one condition is monitored by automaticallyanalyzing video data obtained from said 360 degree video camerautilizing anomaly detection in association with machine learning and GPSdata obtained from said system.
 15. The system of clam 1 wherein said atleast one condition external to said vehicle comprises a pedestrian andsaid change in said at least one condition external to said vehiclecomprises a movement of said pedestrian.
 16. The system of claim 1wherein said signal is transmitted wirelessly to said computing devicethrough a bidirectional wireless communications network.
 17. The systemof claim 1 wherein said at least one condition external to said vehicleis monitored by at least one sensor and at least one video camera. 18.The system of claim 8 wherein said camera comprises a wireless videocamera.
 19. The system of claim 1 wherein said computing devicecomprises an in-vehicle system that provides an alert indicative of saidchange in said at least one condition external to said vehicle aftersaid transmission of said signal to said computing device, saidin-vehicle system associated with said vehicle.
 20. The system of claim1 wherein said monitoring said at least one condition external to saidvehicle, further comprises determining how far said at least onecondition is from said vehicle.